-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC for module redesign. #2108
RFC for module redesign. #2108
Conversation
build layer, instead of the source layer. This has some disadvantages, in that | ||
users may prefer to not have to open the build configuration either. | ||
|
||
This will require migrating users away from `mod` statements toward the new |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will also make building "modern" Rust programs much harder without cargo - you can no longer hardcode the list of dependencies in a variable and just call rustc
on lib.rs
.
Also, might run against the command-line limit for huge crates, but I'm not sure how much of a concern is that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will also make building "modern" Rust programs much harder without cargo - you can no longer hardcode the list of dependencies in a variable and just call rustc on lib.rs.
I'd be eager to get user reports from people actually not passing --extern
arguments today. We can also support an analogous modules path env var.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At least I often collect the --extern
arguments in an env var and call rustc
explicitly when debugging (that includes running rustc
under gdb
and running rustc
under valgrind
, which aren't easy from cargo).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While I don't expect normal users to be debugging rustc, and you can always copy-paste the arguments using cargo build --verbose
, that's one annoyance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, running cargo build --verbose
, editing the argument line, then rebuilding. This will get much much more annoying if argument lines contain a hundred files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like having the option to use a file or an environment variable to specify external dependencies may resolve these concerns.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't cargo support a way to change how it invokes rustc? Why doesn't that work for running rustc under a debugger?
Because with gdb you have to run it as
$ gdb $RUSTC
gdb> set args ARG1 ARG2
gdb> run
$
Also, sometimes editing the command-line to remove some flags, which is sometimes needed when poking around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't gdb --args $RUSTC ARG1 ARG2
work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe it does?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arielb1 If rustc would also allow you to pass a path to a file listing modules to include, would that appease your concerns?
This is how C++ modules work. There one has a modules.modulemap
file, that maps files and sets of files to modules. The compiler can take many of these files, and will then include all modules listed in those files (but they can also include specific files directly).
Since I don't like having to include modules in the Cargo.toml
file (I find this "unclean"), maybe we could adopt a similar approach to C++, where we have a canonical modules.modulesmap
file in the crate directory (just like build.rs
), and if it's there cargo detects it and uses it, and if not, cargo will generate one automatically for you from the filesystem structure in the temporary build directory [*]. That way, you would be able to copy-paste the command of cargo build --verbose
, those wanting to specify the module file hierarchy differently can still write their own .modulemap
files, and my OCD is happy as well because Cargo.toml
remains clean.
[*] we can also allow specifying paths to multiple module map files in Cargo.toml
, but experience with build.rs
has shown that we probably should try if having a canonical location for it is enough. Since modules.modulemap
files can specify paths to other modules.modulemap
files, worst case users would need to add a dummy file that points somewhere else, and we don't need any Cargo.toml
property for this at all.
EDIT: the module.modulemaps
file in C++ allow also specifying things like item-level visibility and stuff like that. It might be worth to explore if allowing something like this in Rust could be used to remove boilerplate. As syntax we can use JSON or TOML to just specify a map of relative file paths to modules or modulemap files. But these things are details.
0000-modules/overview.md
Outdated
`exported`. | ||
|
||
If a user wants to make one of their file submodules a part of their API, they | ||
can do so using the `export keyword (no `use`), followed by the name of the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing a closing quote after export
crates which are both managed by cargo, as when cargo detects that it will | ||
instead pass no `--module` arguments. | ||
|
||
### Using `export` changes semantics |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sounds like a way to annoy people during the transition period. Is there a reason we can't have:
export | pub | pub(crate) | |
---|---|---|---|
today | error | global | crate-local |
checkpoint 1 | global | global (deprecated) | crate-local |
checkpoint 2 | global | crate-local | crate-local (deprecated) |
checkpoint 3 | global | crate-local | error |
The disadvantage is that in checkpoint 1 people will have to mark all of their pub
imports as either pub(crate)
or export
, and they might be annoyed by pub(crate)
being ugly. I think that's better than having pub
mean export
in all "old" crates and pub(crate)
in all "new" crates, with no way to incrementally make the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, it might be nice to have the private-export lint only start in Ckp2, so that in Ckp1 you could just change all pub
to export
and not have to go over them one-by-one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@withoutboats and I talked about such a transition strategy, but:
- It means that code must undergo multiple periods of churn (changing to
pub(crate)
and then topub
) - That's particularly painful for apps, where the distinction doesn't matter
- It greatly prolongs the overall transition
On the other hand:
- we can provide a
rustfix
to automatically convert to the newpub
/export
model, to make it one painless jump - I think in practice it will be highly obvious when you encounter code that is using the old
pub
semantics, and that withrustfix
the transition can and will happen pretty quickly.
Another option, as @withoutboats mentioned, is to move to a different keyword altogether, like public
. I for one would be sad to see pub
go :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't mind seeing pub
go, but I do think its very valuable that if a stack overflow example happens to have pub
in it, it will still compile - in cases like that (e.g. documentation), the semantic change here hopefully doesn't matter very often (that is, I hope you aren't copying your external API from old stack overflow answers 😉).
In other words, even though this "change semantics if we ever encounter extern
" sounds like the most disruptive mechanism possible, when I played out the scenarios in my head, I found that it seemed like the least disruptive thing we could do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's some existing precedent here with pub(restricted)
. Using pub(restricted)
anywhere made the "public in private" warning a hard error (see here).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's a breaker, just annoying when you have a project of 20 crates, half of them use the old semantics, and half use the new semantics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if it's possible to use crate
keyword (currently only used in extern crate
) instead of pub
. This would avoid issues caused by changing well established meaning of pub
keyword. Say...
crate fn hello() {
println!("Hello, world!");
}
crate struct Example {
crate a: i32,
b: i64,
}
This item is expected to be an array of strings, which are relative paths from | ||
the cargo manifest directory to files containing Rust source code. If this item | ||
is specified, cargo will canonicalize these paths and pass them to rustc as the | ||
`--module` argument when building this target. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to clarify-- if any module list is passed here, Cargo won't do any directory searching, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes
One advantage of the |
My position is pretty much the same - I'm for the relative path changes, somewhat for the visibility overhaul (it makes the right thing prettier, but requires an ugly long checkpoint period), and against automod (the |
|
||
// This is an error: you have exported something which is only marked `pub` | ||
export foo::Foo; | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify, this should work:
mod foo {
export struct Foo;
}
export foo::Foo;
but removing export foo::Foo;
would cause a hard error. Is that correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this still apply if the mod foo is in a separate file? This seems confusing to me, that I have to declare Foo as 'exportable' before I can really export it, unless it's being declared at the path where it's being exported, in which case declaring it as exportable is the same as exporting it.
Also in actually writing this code, I wouldn't be able to add 'export' to the 'struct Foo' declaration without also adding the export foo::Foo, and vice-versa. This seems bad for someone who's trying to add in the right export to make their library compile.
Allowing "re-exports" to export items that aren't already exported, and requiring that export statements only appear within the top-level module or explicitly-exported submodules, would take care of both of those confusions. Using export in say a struct or function definition would be shorthand for declaring the struct with some appropriate visibility modifier, and then re-exporting it. Otherwise export just defines the public API of the crate. (And in order to avoid spaghetti code, re-exports don't make symbols that can be used internally by the crate.)
We could make the error on a non-exposed `export` item a warning instead of a | ||
hard error. | ||
|
||
[migration]: detailed-design/migration.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Incorrect link, it should be [migration]: migration.md
.
Something I'm not clear on is whether, with this RFC, |
An unmentioned alternative for file loading is a glob specifier ( |
This is an unresolved question for me! I still don't think it was the right choice, but I'm not sure (especially since the
Possible in macros 2.0 we could even replace |
After a first read I quite like this proposal! I think it's worth to explicitly state that Also how will internal tests modules look like? Today we can write: #[cfg(test)]
mod tests; And |
(edited for clarity and a suggestion) My main issue with the (overall very good) proposal are these two points:
What I would much prefer to see: Refering to external items always requires the This makes it unambiguous that // BEFORE:
use std::collections::HashMap;
type T = ::std::collections::HashMap;
use self::submod::SomeType;
// AFTER:
use ::std::collections::HashMap;
type T = ::std::collections::HashMap;
use submod::SomeType; I reckon this is harder/impossible to do in a backwards-compatible manner? |
I'd also note an addition to the Loading Files Drawbacks: The workflow where you simply comment out module trees during refactoring would become a lot more work. In this scheme, I'd have to maintain a manual module list directly in A flag inside each module (a solution that comes up usually) would require opening every file. I would also assume the file would need to parse correctly for that. |
@phaylon That's also closely related to another issue: platform-specific modules. Without the Both of these issues seem like things that could potentially be solved via some kind of |
Last issue (for now ;)): Another Loading Files alternative is to have auto loading of modules, but a Personally I'd prefer that option to be off by default and activated for new projects in |
I am concerned about a) the amount of breakage and b) the potential confusion introduced by Would it perhaps be better to stick with the current semantics for This notably still keeps the advantages of relative paths for re-exports, via the It does have the downside of potentially making Overall, I believe this removes the dependence on an epoch boundary? Everything can be transitioned piecemeal- loading files by removing |
|
||
```rust | ||
use crate::foo::Bar; | ||
::crate::foo::Bar; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be crate::foo::Bar;
? (Or maybe crate::foo::baz();
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An unresolved question :-)
crate::
matches self::
and super::
. Maybe ::self::
and ::super::
should be valid (and crate should accept both forms also).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, provided this were the case, then in each module, the root ::
would contain external crates, this crate, this module, and this module's parent? One nice thing about this would be that it would mean you could say "use
paths are always relative to ::
", rather than needing the exception "unless they begin with crate
, self
, or super
".
@@ -0,0 +1,194 @@ | |||
# Loading Files | |||
|
|||
When building a Rust project, rustc will and parse some files as Rust code in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will and parse
I'm glad to see that EDIT: Oh I hadn't realised it already exists. |
One thing I could not find mentioned anywhere in the RFC is how to reexport stuff internally. For example, if you have a folder with lots of files, you might want to reexport those under the same module to avoid typing the full path to dozens of different modules. At the same time the module might not be exported and visible in the API. If you deprecate |
@rpjohnst is correct, though I think that's been made obvious enough in the RFC. |
I don't believe this RFC can be accepted in a piecemeal fashion, because these changes interact in many ways. The advantages that we see in this system - incremental learning, heightened local reasoning, ergonomics for the common cases - all emerge from the cumulative impact of these changes together. |
FWIW, I think |
@glaebhoerl I agree, but would much rather have a shorter keyword than the most intuitive one. In fact, I rather like @est31's |
And I still like |
What about |
We can also have |
On a related note, I was thinking about |
I feel bad for thinking it, but if Rust ever wanted to earn some street e-cred you could call it |
How is there not a suggestion to use |
I like it. There might be concern that it doesn't inherently answer the question "visible to whom?" which is the whole point of a visibility modifier. This might make the Another one for the pile: "endo". |
What's the reason |
I just want to say that I find the And to not let an opportunity for bikeshedding go to waste, I wonder if |
@phaylon As in "our stuff"? |
Also, I'm surprised nobody has mentioned |
@mark-i-m Yea. As in |
I don't like that modules are crate visible by default. Yes, it makes it easier to write simple applications, but at some point the user wants a private module (e.g. moving large internal functions to a submodule). At this point it gets much more difficult:
To me it violates both the private-by-default and the explicit-instead-of-implicit principles of Rust. Also, it makes hiding implementation details in private modules (the better design) more difficult than just exposing everything, which I think is also against Rust's principles. |
This also needs to come with a lint that disallows implicit
Splitting the RFC in multiple ones may make it apparent that these advantages you are seeing are marginal and don't justify that many breaking changes for pretty much all code in existence. |
Even though this thread is already closed, I would like to thank everyone involved in the discussion for the work they have put into this. Especially a big "Thank you!" to @aturon on @withoutboats. The outline of the new proposal addresses all the concerns I had with the previous one, and in particular assuages my fears that this RFC could fracture the ecosystem. |
So now that I finally found the time to read through all of this, it is already deprecated: this community is indeed moving fast. Jokes aside I really like the direction this is taking: the first blog post had me aggravated, the second left me still a bit nervous and finally this RFC had me saying „I can live with that“. The new, even more conservative proposal seems even better to me. My only concern with the module system was the whole lot of boilerplate you have to write in order to get names into scope: extern crate, mod, use … I always felt a single To bikeshed a bit, Also I'd still prefer the |
I second the call for the new RFC to be several RFCs. I don't have many issues with the publicity (introducing Adding implicit loading will add quite a few maintainability problems (especially in multi-person projects). E.g. Separate checkouts will have strange errors due to missing paths/impls instead of just pointing to the The current model is generally suitable and matches the eventual code structure well, and I don't recall any real difficulties learning it (but that's just me). |
The new proposal is up! I know this has been a long slog for everyone, but it would be really helpful to get feedback afresh, even if you were already a fan of this RFC. |
Cross-posting from the new thread: A general request: the proposal here appeared to be the consensus direction of the previous thread, but there seems to be significant pushback on this thread. Could those of you who were participating on the previous thread comment saying:
I think one of the primary problems in discussion here tends to be some disagreement about goals. This updated proposal works hard to retain the original goals around learning curves, while taking on some new constraints:
All of these were concerns raised over the course of the last few threads, and I believe this design addresses them while, again, retaining the original benefits. I'd like to understand whether there are additional constraints people feel are missing, or other major downsides to the overall approach that haven't been surfaced yet. (And of course, if you're happy with this approach, it'd be helpful to hear from you too!) |
@aturon Maybe it is because I haven't had the time to let the last proposal sink in me yet, too much information in a too short span of time. My feeling about this has been that I liked the first proposal. It had significant breaking changes, but I thought they were worth it, and that's what epochs are for. I didn't like the second proposal that much, because I am not sure if the wins outweigh the cost. The third proposal moves further along this direction, and I need more time to properly weigh in if they are worth it. The good thing is the third proposal is backwards compatible, so we might want to pursue this now, and after some epoch go to something more drastic like the first proposal. Anyhow the RFC's are long, going through the three of them is a bit tiring, and the stress of willing to move them to FCP ASAP doesn't really help. |
@aturon the new proposal looks much better to me. I think that adding an option to disable auto-scan of modules would help at least during transition (if I have some project with "sketches" - .rs files that are not compile-able, I'd like to be able to compile the project quickly without adding Another thing is that I may want to retain those sketches in my project but keep them private - outside of version control and outside of any public files. Having some private setting to not include them would be helpful. |
@est31 Ah, I missed that, thanks! I like that approach much more than auto-scanning. (but I was under the impression that auto scanning is a highly desired feature) |
Its not off the table yet... the RFC I've linked is only intended to extract all the things that there is a broad consensus for and to get it implemented during the impl period. There is another RFC planned to experiment with stuff (can't provide a link as its not opened yet). |
This RFC describes a proposal to change the Rust module system. Its the culmination of a long, iterative cycle of design and feedback. We, the authors of this RFC (@aturon and myself), believe it is a significant clarification of our current system, making it easier to use and easier to learn.
Because it is a relatively large RFC, I have broken the text into multiple files. You can find the rendered root of the RFC here.
The overview document gives a high level overview of the proposed new system and how it differs from the current system.
For people who have been involved in the iteration process on the internals thread, a brief summary of how it differs from the most recent proposals:
mod
statements; the exact mechanism has been refined & improved from the previous iteration.crate::
rather than some of the more radical syntaxes proposed in recent iterations.For people joining the conversation about modules for the first time, welcome! There has been a lot of iteration on the internals forum which you can read if you want, and there are blog posts (referenced in the RFC) which describe past stages of our thinking about modules. Please don't feel that you do not have enough context to participate in this thread, your feedback matters!
Based on past experience, I predict this RFC will be high traffic. So a couple of reminders:
Lastly, if you want to leave a comment on the RFC but for any reason at all don't want to leave it publicly, I receive email addressed to woboats@gmail.com. I can't promise I'll respond, but I will read it.